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ABSTRACT 

This  paper  summarizes  the  results  of  a  study  forecasting  the  development  of  information  fusion  technology 
over  the  next  20  years  [1].  The  study  was  carried  out  on  behalf  of  the  Swedish  Armed  Forces,  as  part  of 
an  effort  to  estimate  future  developments  in  some  technology  areas  of  high  relevance  for  the  development 
of  a  Network-Based  Defence  system. 


1.0  INTRODUCTION 

Information  fusion  is  still  in  the  research  stage  and  it  is  not  yet  clear  how  its  basic  methodology  should  be 
structured.  The  study  describes  the  current  situation  based  on  a  thorough  and  critical  analysis  of  the  open 
research  literature.  Based  partly  on  published  assessments  by  other  authors,  partly  on  the  author's  own 
experience,  an  estimate  is  then  made  of  the  likely  progress  in  this  area  over  a  10  and  a  20  year  period. 

The  combined  effect  of  rapid  technical  development  in  sensor  technology,  the  establishing  of  new 
communication  technologies  globally,  and  the  literally  exploding  improvement  of  the  price-performance 
ratio  for  computer  systems,  forms  the  technological  background  of  the  efforts  by  a  growing  number  of 
nations  to  create  a  Network-Based  Defence  (NBD). 

This  move  towards  NBD  implies  that  military  decision  makers  will  receive  an  ever  increasing  information 
flux  to  try  to  cope  with.  Information  fusion  is  about  providing  support  in  the  task  of  weighing  together  and 
interpreting  this  information,  i  e,  to  instantiate,  perhaps  even  build,  good  models  of  a  complex,  uncertain 
reality  more  or  less  in  real  time. 

New  and  rapidly  improving  capabilities  to  automatically  collect  large  sets  of  data,  and  to  move  large  sets 
of  data  across  communication  networks,  have  raised  new  questions  regarding  opportunities  to 
automatically  sort,  classify,  and  interpret  data  to  form  an  operational  picture.  In  this  process,  data  from 
various  sources  are  used:  sensor  data,  as  well  as  data  on  probable  behavioral  patterns  of  the  opponent, 
cultural  and  geographic  characteristics  of  the  area  of  operations,  etc.  Such  data  have  previously  been  fused 
to  an  operational  picture  through  time-consuming  manual  analyses  and  discussions.  As  the  availability  of 
sensor  data  explodes  as  a  result  of  technological  advances,  this  manual  fusion  process  becomes  a 
bottleneck  when  establishing  the  operational  picture.  If  sensor  data  can  not  be  processed  and  automatically 
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fused  with  information  already  known  from  other  sources,  the  increased  availability  of  sensor  data  in  the 
widest  sense  will  be  of  little  use.  Also,  it  is  only  through  the  use  of  computerized  information  processing 
that  end  users  can  be  made  to  benefit  from  the  quality,  robustness,  and  conceptual  precision  of  the 
methods  being  developed  by  information  fusion  research. 

Like  in  other  large-scale  and  ambitious  applications  of  information  technology  there  may  exist  a  risk  of 
user  alienation  if  systems  become  too  autonomous  and  do  not  communicate  adequately  with  its  users 
during  the  analysis  and  forecasting  processes.  This  risk  is  closely  related  to  an  increasing  need  for  a 
mathematical  and  scientific  education  of  those  intelligence  officers  who  will  be  the  users  of  future 
information  fusion  systems. 


Table  1:  General  Forecast  for  the  Development  of  Automated  Information  Fusion 


Situation  in  2002 

Estimated  situation  in  2012 

Estimated  situation  ca  2022 

Immature  technology. 
Methodological  basis  of 
information  fusion  not  yet 
established. 

Methodology  for  situation 
analysis  developed  for  many 
military  applications. 

Effective  systems  for  threat  and 
situation  analysis  integrated  in 
NBD. 

2.0  TECHNOLOGY  AND  SYSTEMS  FOR  INFORMATION  FUSION 

Information  technology  -  primarily  software  -  supporting  the  task  of  creating  in  real  time  a  correct 
situation  description,  or  more  precisely,  model,  adequate  for  the  decision  support  needs  of  an  operative  or 
tactical  military  commander,  will  be  called  here  information  fusion  systems.  The  concept  covers  those 
parts  of  the  information  fusion  process  which  are  intended  to  support  analysts  and  decision  makers  in  their 
planning  and  execution  of  intelligence  collection,  as  well  as  their  analysis,  of  intelligence  from  all  sources, 
i  e  processes  which  are  more  closely  related  to  intelligence  processing  than  to  sensor  data  processing  or 
target  position  estimation. 

The  purpose  of  developing  and  deploying  systems  for  information  fusion  in  tactical  and  intelligence 
systems  is  thus  to  provide  support  to  commanders  and  intelligence  officers  in  their  continuous  work  of 
collecting,  processing,  and  conceptualizing  information  about  various  potential  threats  during  an  ongoing 
military  operation.  Sensors  and  multi-sensor  systems  have  important  roles  as  "subcontractors"  in  this 
process,  however,  in  focus  here  is  the  issue  of  how  to  shape  computer  supported  methodology  for  real¬ 
time  modelling,  i  e,  representation,  estimation,  interpretation,  and  forecasting  of  threat  extent,  structure, 
capability,  and  options,  given  what  is  known  about  the  opponent's  situation  and  behaviour,  or  doctrine,  as 
well  as  the  geographical  environment.  Note  that  the  intent  of  the  opponent  will  usually  be  unknown,  even 
"unknowable",  to  the  fusion  system,  whose  task  is  instead  to  establish  and  evaluate  all  his  options  to  act. 
The  underlying  idea  is  that  from  a  sophisticated  analysis  and  evaluation  of  these  options  using 
mathematical  modelling,  it  should  usually  be  possible  to  radically  but  rationally  reduce  the  number  of 
likely  enemy  actions  to  a  small  set,  manageable  by  human  decision  makers,  and  whenever  this  is  found  not 
to  be  the  case,  to  redirect  in  real  time  one's  own  intelligence  collection  resources  so  as  to  eventually  reach 
that  goal. 

Reconnaissance  and  analysis  resources,  as  well  as  fusion  results  such  as  situation  and  forecasting 
information,  must  be  accessible  from  all  levels  in  the  operative  organisation.  In  dialogue  with  its  users,  the 
system  needs  to  be  able  to  make  good  proposals  to  resolve  resource  conflicts  as  they  occur.  Also,  it  needs 
to  function  in  situations  when  communications  have  low  capacity,  even  when  they  are  intermittent, 
although  necessarily  with  reduced  precision  of  its  result.  Thus,  the  feasibility  of  information  fusion 
depends  strongly  on  the  availability  of  a  seamless  and  flexible  information  network.  Without  such  a 
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network,  neither  input  to  nor  output  from  fusion  processes  will  reach  their  destination  in  time. 
Quantitatively,  the  message  delivery  time  and  bandwidth  of  each  link  in  the  network  at  each  point  in  time 
will  be  important  factors  influencing  the  total  capability  of  the  information  fusion  system. 


3.0  INFORMATION  FUSION  TERMINOLOGY  AND  PROCESSES 

Language  usage  in  the  fusion  field  has  been  and  still  is  varying,  so  that  there  is  reason  to  define  the 
terminology  used  in  this  paper  (and  in  the  Swedish  R&D  community  for  NBD). 

Sensor  data  fusion,  or  using  a  practical  abbreviation,  sensor  fusion,  deals  with  creating  a  target  situation 
description  (or  "picture")  based  on  data  received  from  the  sensors.  Static  data,  such  as  sensor  and  target 
properties  or  threat  libraries,  are  also  sometimes  used.  Sensor  fusion  primarily  involves  target  tracking, 
but  also  target  identification  based  on  sensor  data.  Multi-sensor  fusion,  in  particular,  denotes  such  sensor 
fusion  where  more  than  one  sensor  is  employed  to  create  target  tracks  or  situation  picture.  The  purpose  of 
multi-sensor  fusion  is  to  combine  information  from  several  sensors  in  order  to  obtain  greater  robustness, 
precision,  and  range  from  the  sensor  systems.  Multi-sensor  multi-target  fusion  is  sometimes  used  when 
one  wants  to  emphasize  the  use  of  a  set  of  several  sensors  to  create  and  maintain  a  target  situation  picture 
containing  many  different  perceived  targets  which  may  perhaps  also  appear  and  disappear  randomly. 

Information  fusion  employs,  in  addition  to  all  available  target  information,  other  kinds  of  intelligence  data, 
as  well  as  other  relevant  kinds  of  data.  This  is  all  fused  during  situation  assessment  into  a  higher-level 
situation  model  (here  too  sometimes  called  "situation  picture",  a  usage  we  have  frequently  found 
misleading).  Interpretation  of  the  situation  -  threat,  vulnerability,  tactics  etc.  -  is  performed  in  the  impact 
assessment  process.  The  resource  management  process  provides  feedback  to  the  sensors  as  well  as  to  the 
assessment  processes.  In  our  terminology.  Levels  0  and  1  in  the  so-called  JDL  model  do  not  belong  to 
information  fusion  and  should  be  viewed  as  sensor  fusion  processes. 


Figure  1:  JDL  Model. 


3.1  Level  2:  Situation  Assessment 

Results  from  sensor  fusion  processes  (level  0  and  1)  are  passed  to  a  decision  support  system  for  situation 
and  impact  assessment.  During  situation  assessment  a  model  is  reconstructed  of  the  situation  which  caused 
the  observed  data  and  events.  A  set  of  alternative  hypotheses  about  the  perceived  situation  is  also 
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generated.  The  hypotheses  are  evaluated  with  respect  to  the  observations  and  are  ascribed  probabilities. 
The  goal  of  this  analysis  is  to  find,  characterize,  and  rank  the  most  likely  hypotheses.  Situation  assessment 
is  a  chronologically  ordered  process  where  data  arrive  over  time  and  the  analysis  is  successively  refined 
towards  a  gradually  more  detailed  and  probable  best  hypothesis. 

3.1.1  The  Aggregation  Problem 

If,  like  during  situation  assessment,  one  strives  to  understand  an  ongoing  operational  development, 
the  relevant  level  of  analysis  is  usually  not  separate  physical  objects  but  groups  of  organizationally  related 
objects.  Frequently,  the  goals  of  a  military  operation  are  such  that  no  single  unit  is  indispensable  for 
completion  of  the  task.  Further,  a  group  of  platforms  may  present  a  different  -  and  much  greater  -  threat 
than  the  sum  of  all  the  threats  presented  by  each  individual  platform.  Thus,  in  addition  to  potentially 
meeting  the  need  of  the  decision  maker  to  have  the  situation  presented  in  a  way  which  avoids  too  much 
detail,  aggregation  of  vehicles  or  other  platforms  to  military  units  promises  to  be  an  important  step 
towards  the  goal  of  unmasking  the  opponent's  intent  and  find  effective  countermeasures. 

The  aggregation  problem  for  sets  of  objects  [2]  has  certain  similarities  to  the  classification  and 
identification  issues  for  single  objects  which  are  treated  during  object  assessment.  Based  on  observations 
of  single  objects  (or,  more  accurately,  of  certain  attributes  of  an  object),  one  wants  to  deduce  which  kind 
of  unit  is  observed.  The  difficulties  are  in  some  respects  similar.  How  do  you  determine  if  two  or  more 
observations  belong  to  the  same  unit  (object)?  What  kind  of  a  priori  knowledge  do  you  need  to  determine 
which  unit  (object)  the  various  objects  (attributes)  belong  to?  How  should  this  a  priori  knowledge  be 
represented,  how  should  the  evaluation  be  carried  out,  and  how  do  you  best  represent  the  uncertainty  of 
the  conclusions? 

In  the  aggregation  problem  there  is  the  added  complication  that  unit  types  belong  to  some  (usually 
hierarchical)  organizational  structure.  Different  objects  may  belong  to  different  units,  different  units  may 
belong  to  different  companies,  etc.  Certain  identification  on  one  level  may  be  combined  with  uncertain 
identification  on  another  level.  Also,  the  structures  at  each  level  are  dependent  of  each  other.  Higher  units 
may  contain  certain  specific  smaller  subunits  and  at  the  same  time  some  subunits  may  in  themselves  be 
organization-determining.  Methods  which  can  deal  with  such  dependencies  are  therefore  likely  to  be 
particularly  interesting. 


3.1.2  Level  3:  Impact  Assessment 

Impact  or  threat  assessment  is  a  process  which  should  be  viewed  from  several  perspectives. 

Here  risks  and  opportunities  are  analyzed  for  own  forces  to  meet  the  opponent  in  an  effective  manner. 
Results  from  the  previous  situation  assessment  is  combined  with  received  indications  about  the  opponent's 
intent  as  well  as  prestored  information  from  technical  and  doctrinal  databases  to  deliver  a  complete  impact 
assessment. 

In  impact  assessment  possible  threats  are  identified  and  ranked  according  to  the  character  and  time  horizon 
of  the  threat.  Until  now,  research  has  largely  been  focused  on  air  and  sea  applications  where  the  threat 
analysis  takes  its  departure  from  own  aircraft  or  ship  survival.  In  simpler  cases  the  threats  from  adverse 
forces  are  evaluated  based  on  distance  and  speed  of  approach  of  the  threat.  Here  one  also  needs  to  take  the 
speed  of  own  aircraft/ship  into  account.  Doctrinal  data  play  an  important  role.  Knowledge  about  weapon 
systems  and  their  ranges  may  help  refine  threat  evaluation,  and  more  complex  doctrinal  data  on, 
e  g,  tactical  behaviour  might  also  be  used  in  the  modelling. 

In  a  longer  time  perspective  the  threat  analysis  may  be  extended  from  dealing  only  with  the  security  of  the 
own  unit  to  that  of  other  targets  of  interest  to  the  opponent.  Here  too  doctrinal  information  about  the 
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opposing  units  will  play  an  important  role,  however,  needs  to  be  extended  with  geographical  data  about 
avenues  of  approach  and  trafficability. 

To  estimate  the  probability  that  a  potential  threat  is  real  is  a  critical  part  of  impact  assessment.  Potentially, 
each  enemy  unit  present  within  a  certain  area  a  possible  threat  to  any  valuable  targets  in  the  area,  but 
obviously  certain  combinations  of  units  and  threats  are  less  likely.  The  generation  of  threat  models 
therefore  needs  to  be  sensitive  to,  and  provide  a  representation  of,  those  factors  which  are  likely  to 
influence  the  probability  assessment  of  various  threats,  such  as  distance  to  target,  direction  of  movement, 
military  strength  and  type  of  unit. 

3.2  Level  4:  Adaption  and  Resource  Allocation 

Multisensor  systems  become  ever  more  important  as  a  component  in  a  number  of  applications,  military  as 
well  as  civilian.  Since  a  single  sensor  can  usually  perceive  only  limited  and  partial  information  about  its 
environment,  one  needs  to  make  use  of  several  coordinated  sensors  of  the  same  or  different  types, 
to  obtain  sufficiently  many  comprehensive  local  views  with  different  focus  and  from  different 
perspectives,  in  order  to  create  a  complete,  integrated  view  of  a  complex  phenomenon  [3].  This  presumes 
that  an  underlying  model  of  the  observed  phenomenon  exists  and  may  be  exploited.  The  value  of  multi¬ 
sensor  systems  therefore  consists  in  their  ability  to  create  greater  opportunities  for  machine  perception  and 
to  improve  the  perception  of  the  state  of  the  observed  environment,  compared  to  what  could  be  achieved 
by  a  single-sensor  system. 

On  a  higher  level,  sensor  management  deals  less  with  direct  control  in  real  time,  although  this 
functionality  may  well  be  desirable  in  the  future,  than  how  best  to  plan  the  use  of  (combinations  of) 
mobile  sensor  platforms  such  as,  e  g.  Unmanned  Aerial  Vehicles  (UAV).  The  basic  issues,  however, 
remain  the  same:  sensor  resources  are  to  some  degree  given,  however,  the  information  needs  have  to  be 
defined.  Here,  impact  assessment  plays  an  important  role.  In  the  simplest  cases  the  issue  is  how  to  cover  a 
set  of  areas  in  the  most  efficient  way.  Since  maybe  not  all  areas  can  be  covered  certain  areas  will  have  to 
be  prioritized  with  respect  to  current  threat  patterns. 

In  dedicated  sensor  systems  resource  allocation  essentially  consists  of  direct  sensor  control:  how  are 
sensors  to  be  controlled  in  order  to  track  a  target  or  cover  an  area  in  the  most  efficient  way.  Basically,  this 
is  an  optimization  problem  where  sensor  resources  are  weighed  against  information  needs.  Sensor 
resources  are  given,  but  the  information  need  may  be  defined  in  different  ways. 


4.0  RESEARCH  STRATEGY 

The  key  research  strategy  issue  in  this  context  is  to  identify  problem  areas  and  specific  research  issues 
which  can  be  tackled  by  a  coherent  methodology,  and  at  the  same  time  have,  or  can  be  expected  to 
acquire,  tactical,  operative,  or  strategic  relevance. 

Although  fusion  methodology  is  of  great  interest  in  a  number  of  application  areas,  we  focus  here  on 
military  applications.  This  characterization  is  however  far  from  sufficient  when  addressing  the  question 
whether  information  fusion  is  a  possible,  respectively  an  important,  technology  in  network-based  defence. 
Those  problems  which  may  need  fusion  support  during  an  international  peace-keeping  operation  are 
different  from  those  which  need  to  be  solved  when  defending  against  an  armed  attack  on  one's  own 
territory.  In  another  dimension,  issues  of  strategic  intelligence  during  peacetime  are  largely  different  from 
those  which  need  to  be  answered  at  the  tactical  level  in  wartime.  Applying  information  fusion  to  the 
defence  of  a  country's  infosphere  is  again  different  from  information  fusion  applied  to  the  forecasting  of 
an  enemy  army  unit's  movements  and  intents. 
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This  does  not  imply  that  there  is  no  ground  for  considering  all  these  problems  as  related,  and  by  doing  so, 
assigning  the  information  fusion  concept  a  broad,  cross-disciplinary  meaning.  It  does  imply,  however,  that 
the  technical  forecast  we  are  trying  to  produce  has  to  be  made  with  specific  applications  in  mind. 

Although  it  is  obviously  impossible  in  general  to  foresee  with  certainty  what  an  enemy  unit  is  going  to  do, 
on  the  other  hand  no  commander  of  a  ground-based  unit  has  much  flexibility  to  rapidly  change  the 
situation  of  his  unit.  The  emergence  of  increasingly  swift  observation,  command,  and  impact  processes 
will  therefore  lead  to  an  increased  need  for  tools  that  can  provide  timely  and  accurate  short-time 
predictions. 

However,  to  be  able  to  make  longer-term  forecasts  than  those  purely  kinematical  and  dynamical 
predictions  which  multisensor  fusion  provides,  one  has  to  exploit  two  complex  scientific  areas: 

•  quantitative,  or  in  some  cases  qualitative  theories  for  reasoning  under  uncertainty 

•  knowledge  about  those  limitations  which  the  environment  and  the  organization  and  doctrine  of  the 
enemy  impose. 


5.0  HOW  CAN  INFORMATION  FUSION  BE  AUTOMATED? 

In  Handbook  of  Multisensor  Data  Fusion  [4],  this  issue  is  discussed  by  Hall  och  Steinberg. 

5.1  Situation  Assessment 

Improved  methodology  for  situation  assessment,  a  central  information  fusion  subject  area,  considered  by 
these  authors  to  be  immature  today,  will  probably  require  a  better  understanding  of  how  to  select  and  use 
existing  methods  for  knowledge  representation  (like  rules,  frames,  scripts,  and  fuzzy  logic),  in  conjunction 
with  a  better  appreciation  of  the  strengths  and  weaknesses  of  human  reasoning  about  such  problems. 
One  example  is  the  need  to  employ  reasoning  based  on  negative  information,  i  e  information  that  has  not 
been  observed  but  should  be  expected  in  a  hypothetical  situation.  Another  promising  area  is  considered  to 
be  development  of  tools  for  compensating  known  weaknesses  in  the  way  humans  manage  information 
(e  g,  humans  often  look  for  information  which  can  confirm  a  stated  hypothesis,  rather  than  information 
that  may  refute  it). 

5.2  Impact  (Threat)  Assessment 

Regarding  methods  and  systems  for  impact  assessment  the  situation  is  said  to  be  similar  but  even  more 
immature.  The  problem  of  predicting  an  opponent's  intent  is  a  basic  issue  in  impact  assessment,  and  it  can 
be  systematically  modelled  in  many  cases  when  the  opponent's  behavior  follows  a  known  doctrine. 
In  modem  wars  and  conflicts,  however,  doctrine  is  frequently  poorly  developed,  unknown,  or  even  non¬ 
existent. 

Impact  assessment  is  dependent  on  our  ability  to  recognize  behaviors,  and  furthermore  requires  methods 
for  evaluating  both  enemy  intent  given  a  certain  behavior,  as  well  as  for  modelling  one's  own  vulnerability 
against  various  kinds  of  threats.  Since  interpretation  of  threats  and  possibilities  can  seldom  be  made 
unequivocally,  it  is  important  to  endow  a  system  for  impact  assessment  with  the  ability  to  represent  and 
manage  many  alternative  interpretations,  all  with  some  measure  of  likelihood  attached.  Here,  one's 
dependence  on  doctrinal  knowledge  and  behavioral  mle  systems  is  even  greater  than  during  situation 
assessment.  This  we  consider  to  be  one  of  the  greatest  obstacles  against  the  development  of  methods  that 
can  be  applied  across  a  wide  spectmm  of  unconventional  threat  scenarios.  It  is  likely,  e  g,  that  new 
methods  must  be  created  for  mapping,  and  later  perhaps  modelling,  of  organizational  stmcture  and  threat 
behavior  of  terror  and  guerilla  organizations. 
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5.3  Process  Control 

Process  control  involves,  e  g,  methodology  which  has  long  been  applied  to  controlling  single  sensors  and 
homogeneous  sensor  systems.  In  information  fusion  the  concept  has  a  wider  meaning,  however:  how  can 
the  entire  intelligence  collection  process  be  optimally  controlled  in  real  time?  In  this  area,  there  is  much 
work  left  to  do,  although  significant  progress  can  be  expected  also  in  the  shorter  term. 

6.0  FORECAST 

Our  own  view  of  the  state  of  development  in  the  areas  of  situation  and  impact  assessment  differs  in  a  few 
but  presumably  significant  respects  from  the  one  presented  in  the  cited  paper  by  Hall  and  Steinberg. 
An  important  recent  trend  has  been  the  emergence  of  new  theoretical  approaches,  creating  significantly 
better  opportunities  for  managing  uncertainty  and  ambiguity  in  information  systems.  In  general,  new 
representation  and  analysis  methods  from  the  discipline  called  management  of  uncertainty  are  appearing 
more  or  less  continuously,  some  of  which,  e  g  Bayesian  Networks  and  Abstract  Hidden  Markov  Models, 
promise  to  be  effectively  applicable  to  information  fusion  problems,  such  as  multi-agent  plan  recognition 
[5]. 

In  particular,  we  believe  that  new  developments  in  finite  set  statistics,  new  particle  filtering  methods,  and 
clustering  methods  for  force  aggregation,  in  combination  with  new  methods  for  multisensor  management, 
advanced  simulation  systems  and  terrain  models,  will  lead  to  important  advances  regarding  detection, 
classification,  tracking,  and  prediction  of  complex  set  objects,  such  as  ground  forces  on  various 
organizational  levels.  In  light  of  these  and  other  scientific  advances,  our  assessment  can  be  considered 
somewhat  more  optimistic  than  that  of  Hall  and  Steinberg. 

We  believe  that  the  problems  that  primarily  have  to  be  solved  to  enable  significant  automation  of 
information  fusion  processes  are  in  large  measure  scientific  methodology  issues  about  how  to  formulate, 
model,  and  solve  certain  generic  problems,  whose  structure  is  often  currently  insufficiently  known.  If  one 
recognizes  this,  it  becomes  easier  to  understand  why  research  in  this  area  will  require  considerable 
resources,  including  probably  calendar  time,  before  practical  results  can  be  obtained  across  a  broad 
application  area. 

In  conclusion,  we  estimate  that  within  ten  years  there  will  exist  methodology  for  information  fusion  which 
will  be  useful  in  practice  for  the  Swedish  defence,  in  particular  for  situation  assessment  in  ground  warfare 
scenarios,  and  that  an  additional  10  years  of  focused  research  may  lead  to  the  establishing  of  effective, 
productivity-boosting  such  systems  within  the  context  of  a  network-based  defence.  This  prediction 
presumes  that  the  quality  and  quantity  of  this  research  does  not  diminish  but  instead  successively 
improves  during  the  next  5  to  10  years. 
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The  need  for  information  fusion  automation 

Networked  sensor  and  communication  systems  can 

generate  intense  information  flows  and  wiil  change 

commander’s  main  concern 

■  from  managing  the  “the  fog  of  war”,  i.e.,  vague  and 
incomplete  information 

■  towards  interpreting  and  understanding  of  great 
amounts  of  compiex  information  under  severe  time 
constraints 

Automating  information  fusion  is  therefore  a  critical 

research  task  in  the  quest  for  information  superiority 
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What  is  (tactical)  information  fusion  ? 

■  information  fusion  exploits 

-  available  tactical  intelligence  information 

-  information  about  the  environment 

-  known  properties  of  the  opponent’s 
platforms,  combat  units,  organisation  and 
doctrine 

■  to  dramatically  improve  speed  and  accuracy  of 
assessing  and  predicting  the  opponent's 
capabilities  and  intent 
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Information  fusion  subprocesses  (JDL) 

■  situation  assessment,  e,  g.,  instantiate  a  relevant 
and  adequate  situation  mode/  of  a  tacticai  situation 

•  impact  assessment,  e.  g.,  evaluate  situation  and 
predict  possible  consequences  of  decision 
options 

•  process  refinement,  e.  g.,  use  situation  and 
impact  assessments  in  combination  with  models 
of  information  collection  capabilities  to  find  and 
execute  near-optimal  collection  schemes 
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Where  can  information  fusion  be  used... 

■  in  interpretation  of  observations: 

-  position  and  timing  estimation 

-  classification  of  objects  and  systems 

-  identification  of  objects  and  systems 

-  correlation  of  observations  dispersed  in  time 
and  space 

-  aggregation  to  systems  from  observations  of 
components 
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...Where  can  information  fusion  be  used? 

■  in  assessment  of  the  enemy’s  potential  courses  of 
action: 

-  probable  time  and  place  for  engagement 

-  estimation  of  the  enemy’s  fighting  capability 

-  estimation  of  relative  strengths 

■  in  combination  of  information  from  different 
sources,  including  databases,  sensors,  and 
humans 

■  in  optimization  of  the  intelligence  collection 
process 
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Current  situation  in  information  fusion 
(foliowing  Hall  &  Steinberg,  2001) 

■  many  prototype  systems  have  been  developed,  most 
of  them  restricted  to  situation  assessment 

■  KBS  technology  has  dominated 
Variants  include: 

-  blackboard  systems 

-  systems  based  on  logical  templates 

-  case-based  reasoning  systems 

Emerging  use  of  fuzzy  logic  and  agent-based  systems 
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Situation  assessment,  current  limitations 

■  few  and  limited  operational  systems 

■  experience  lacking  from  upscaling  prototypes  to 
operational  systems 

■  very  limited  cognitive  models 

■  unsystematic  testing  and  evaluation,  on  model 
problems  only 

■  no  widely  accepted  technology  exists  for 
knowledge  management 
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Impact  assessment,  current  limitations 

■  doctrine  specific,  brittle  systems 

■  few  and  limited  operational  systems 

■  experience  lacking  from  upscaling  of  prototypes  to 
operational  systems 

■  quantifying  intent  is  a  difficult  problem 

■  models  require  known  and  established  enemy 
doctrine 

■  quickly  changing  situations  are  difficult  to  model 
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Process  refinement,  current  limitations 

■  hard  to  take  mission  constraints  into  account 

■  scaiahility  problems  for  adaptive  systems  with 
many  sensors 

■  difficult  to  use  non-commensurable  sensors 
optimally 

■  very  difficult  to  tie  sensor  management  to  human 
information  needs 
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Can  we  see  some  progress? 

-  in  situation  assessment:  systematic  uncertainty 
management  {random  sets,  Dempster~Shafer 
theory),  use  of  general  tracking  concept  in  force 
aggregation  and  group  tracking  problems 

■  in  impact  assessment;  new  simuiation  methods, 
e.g.,  ANTS  for  determining  and  evaluating  avenues 
of  approach,  dynamic  Bayesian  networks  in  pian 
recognition  and  intent  assessment  {R.  Suzic,  this 
conference) 

■  in  process  refinement:  game  theory-based 
negotiation  to  support  cooperative  use  of  possibiy 

FOI  non-commensurabie  sensors 
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What  is  force  aggregation? 


■  in  situation  assessment,  the  task  of  estimating 
the  composition  and  structure  of  an  enemy  unit 
from: 

-  a  set  of  observations  of  objects  belonging  to 
the  unit 

-  a  priori  knowledge  about  doctrine  (materiel, 
organisation,  behaviour) 

■  several  approaches  are  possible;  FOI  has 
worked  particularly  with  methods  based  on 
Dempster-Shafer  clustering 
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Force  aggregation  as  clustering  +  classification 


Class  ifi  cation 
of  battalions 


. 

Class  ifi  cation 
of  companies 


Class  ifi  cation 
of  platoons 


Class  ifi  cation 
of  vehicles 
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What  is  tracking  really?  (N.  Gordon  &  ai) 

■  tracking  uses  models  of  the  real  world  to  be  able  to 
estimate  the  past  and  present  and  predict  the  future 

■  this  is  achieved  by  extracting  underlying  information 
from  an  incoming  stream  of  noisy/uncertain 
observations 

Any  extraction  of  information  from  a  time  evoiving 
system  is  a  tracking  problem;  tracking  applications 
are  innumerable. 

Only  recently,  general  tracking  algorithms,  based  on 
particie  fiitering,  have  become  available. 
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Two  military  tracking  challenges 

■  estimate,  in  the  presence  of  clutter  and  physical 
obstructions,  using  spatio-temporal  environmental  cues, 
which  objects  are  present  in  an  area  of  interest,  where 
they  are  now  and  how  they  moved  there  {situation 
“picture”) 

■  identify  and  track  units  on  higher  organisational  levels 
(sets  of  objects),  combining  the  problems  of  tracking, 
aggregation,  and  identification  {tacticai  inteiiigence 
acquisition) 
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Example:  sketchy  preview  of  FOI’s  IFD03 

■  Systematic  architecture  based  on  simulation  framework 

■  Human  observers  and  UAVs  make  uncertain  detections 
and  report  to  a  remote  command  center 

■  Command  center  starts  a  particle  filter  tracking  module, 
which  is  continuously  updated  by  new  reports 

■  Reports  are  aggregated  to  vehicles,  then  to  platoons 

■  4  views: 

Sensors  with  their  reports  Clustered  reports:  vehicles 
Clustered  vehicles:  platoons  Particle  filter  output 
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Fusion  of  reports  in  IFD03... 


■  reports  are  clustered  using  Dempster-Shafer  clustering 
techniques 

■  the  force  aggregation  service  generates  aggregation 
results  based  on  available  sensor,  terrain,  and  doctrine 
information 

■  lower  units  are  aggregated  to  higher  and  are  classified 
whenever  sufficient  information  is  available 
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...Fusion  of  reports  in  IFD03 

■  the  particle  filter  termin  tracker  service  Is 
exploited  by  the  sensor  resource  manager  to 
allocate  and  focus  available  sensor  resources 

■  when  an  aggregation  result  (unit)  has  been 
created,  it  is  passed  to  the  particle-filter-based 
group  tracker,  which  repeatedly  reports  the 
unit's  position  and  speed 
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Automatic  information  fusion  -  opportunities 

-  any  sweeping  claim  that  information  fusion  will  be 
capable  of  automatic  prediction  of  enemy  actions  is 
nonsense 


■  however,  in  properly  delimited  frameworks,  behavioral 
simulation  models  based  on  robust  uncertainty 
management  methods  may  be  systematically  fused 
with  observations  to  form  useful  predictions 

■  so,  application  of  systematic  uncertainty  management 
methodology  to  weil-specified  miiitary  situations  will 
likely  create  new  opportunities  for  generating  and 
evaluating  decision  alternatives 
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Conclusion  and  forecast 


■  well-established  sensor  fusion  methods  can  rarely  be 
used  on  a  tactical  intelligence  level 

■  however,  via  formulating  and  solving  a  ’’critical  mass” 
of  concrete  problems  using  systematic  methodologies 
and  measures  of  effectiveness,  a  simuiation-based 
technology  for  information  fusion  is  iikeiy  to  be  created 
in  the  next  ten  to  fifteen  years 

■  reasonabiy  complete,  deployable  systems  will  be 
developed  based  on  such  technology 

■  hard  architectural,  logistical,  and  organisational  issues 
need  to  be  addressed  for  iF  systems  to  be  successfui 
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